Identifying populations at risk: functional impairment and emotional distress

Presentation at the 2002 Medical Director Colloquy, Tucson, Ariz., May 16–18. Title of Managed Care supplement: Multidisciplinary Management of Dyslipidemia

Using bootstrap to compare the validity of PRO measures in discriminating among CKD patients

BACKGROUND: Patient-reported outcome (PRO) research requires valid and sensitive measures. Relative Validity (RV) offers an objective way to compare the validity of different PRO measures in discriminating groups of patients or occasions. There is no significance test associated with RV. We applied bootstrap to estimate the confidence interval (CI) of RV to better interpret the differences in RV. METHODS: The CKD-specific legacy (KDQOL Burden, Symptom, and Effect), generic health scales (SF-12), and Kidney Disease Impact Scale (KDIS) were administrated to 453 CKD patients. ANOVA-based RV coefficients were computed to compare how well each scale discriminated between three clinically-defined severity groups (Dialysis \u3e Stage 3-5 \u3e Transplant). Bootstrap was used to construct CI to determine whether the differences in RV were significant in comparisons between each scale and the best legacy standard- KDQOL Burden. Factors of sample size, number of bootstrap replications, bootstrap method were varied to investigate their impacts. RESULTS: In comparison with KDQOL Burden (RV=1), using 95% CI, differences were non-significant for KDIS (RV=1.13), KDQOL Effect (RV=.99), SF-12 RP (RV=.77) and PF (RV=.70). SF-12 PCS (RV=.60) was at borderline. The other measures were significantly poorer in discriminating the patients. Sample size played a substantial role. 300 patients for 3 groups greatly reduced the standard errors compared to 100 patients. A larger sample size greatly increased the power of detecting the differences. The number of replications did not have consequential influence. The types of BCa and percentile intervals were preferred as all bootstrap distributions were skewed. The magnitude of chosen standard measure’s F-statistics appeared to have a noticeable impact on CI too. CONCLUSIONS: Bootstrapping appears to be valuable in comparing the validity of PRO measures from a statistical perspective. The significance test of RV was affected by the sample size, magnitude of RV, and F-statistic of standard measure

Standardizing disease-specific quality of life measures across multiple chronic conditions: development and initial evaluation of the QOL Disease Impact Scale (QDIS(R))

BACKGROUND: To document the development and evaluation of the Quality of life Disease Impact Scale (QDIS(R)), a measure that standardizes item content and scoring across chronic conditions and provides a summary, norm-based QOL impact score for each disease. METHODS: A bank of 49 disease impact items was constructed from previously-used descriptions of health impact to represent ten frequently-measured quality of life (QOL) content areas and operational definitions successfully utilized in generic QOL surveys. In contrast to health in general, all items were administered with attribution to a specific disease (osteoarthritis, rheumatoid arthritis, angina, myocardial infarction, congestive heart failure, chronic kidney disease (CKD), diabetes, asthma, or COPD). Responses from 5418 adults were analyzed as five disease groups: arthritis, cardiovascular, CKD, diabetes, and respiratory. Unidimensionality, item parameter and scale-level invariance, reliability, validity and responsiveness to change during 9-month follow-up were evaluated by disease group and for all groups combined using multi-group confirmatory factor analysis (MGCFA), item response theory (IRT) and analysis of variance methods. QDIS was normed in an independent chronically ill US population sample (N = 4120). RESULTS: MGCFA confirmed a 1-factor model, justifying a summary score estimated using equal parameters for each item across disease groups. In support of standardized IRT-based scoring, correlations were very high between disease-specific and standardized IRT item slopes (r = 0.88-0.96), thresholds (r = 0.93-0.99) and person-level scores (r \u3e /= 0.99). Internal consistency, test-retest and person-level IRT reliability were consistently satisfactory across groups. In support of interpreting QDIS as a disease-specific measure, in comparison with generic measures, QDIS consistently discriminated markedly better across disease severity levels, correlated higher with other disease-specific measures in cross-sectional tests, and was more responsive in comparisons of groups with better, same or worse evaluations of disease-specific outcomes at the 9-month follow-up. CONCLUSIONS: Standardization of content and scoring across diseases was shown to be justified psychometrically and enabled the first summary measure of disease-specific QOL impact normed in the chronically ill population. This disease-specific approach substantially improves discriminant validity and responsiveness over generic measures and provides a basis for better understanding the relative QOL impact of multiple chronic conditions in research and clinical practice

Predicting declines in physical function in persons with multiple chronic medical conditions: what we can learn from the medical problem list

BACKGROUND: Primary care physicians are caring for increasing numbers of persons with comorbid chronic illness. Longitudinal information on health outcomes associated with specific chronic conditions may be particularly relevant in caring for these populations. Our objective was to assess the effect of certain comorbid conditions on physical well being over time in a population of persons with chronic medical conditions; and to compare these effects to that of hypertension alone. METHODS: We conducted a secondary analysis of 4-year longitudinal data from the Medical Outcomes Study. A heterogeneous population of 1574 patients with either hypertension alone (referent) or one or more of the following conditions: diabetes, coronary artery disease, congestive heart failure, respiratory illness, musculoskeletal conditions and/or depression were recruited from primary and specialty (endocrinology, cardiology or mental health) practices within HMO and fee-for-service settings in three U.S. cities. We measured categorical change (worse vs. same/better) in the SF-36(R) Health Survey physical component summary score (PCS) over 4 years. We used logistic regression analysis to determine significant differences in longitudinal change in PCS between patients with hypertension alone and those with other comorbid conditions and linear regression analysis to assess the contribution of the explanatory variables. RESULTS: Specific diagnoses of CHF, diabetes and/or chronic respiratory disease; or 4 or more chronic conditions, were predictive of a clinically significant decline in PCS. CONCLUSIONS: Clinical recognition of these specific chronic conditions or 4 or more of a list of chronic conditions may provide an opportunity for proactive clinical decision making to maximize physical functioning in these populations

Psychometric evaluation of the SF-36 health survey in Medicare managed care

Data quality and scoring assumptions for the SF-36 Health Survey were evaluated among the elderly and disabled, using 1998 Cohort I baseline Medicare HOS data (n=177,714). Missing data rates were low, and scoring assumptions were met. Internal consistency reliability was 0.83 to 0.93 for the eight scales and 0.94 and 0.89, respectively, for the physical (PCS) and mental (MCS) component summary measures. Results declined with increased risk factors (e.g., older age, more chronic conditions), but were well above accepted standards for all subgroups. These findings support using standard algorithms for scoring the SF-36 in the HOS and subgroup analyses of HOS data

Evaluation of smoking-specific and generic quality of life measures in current and former smokers in Germany and the United States

BACKGROUND: Health-related quality of life (QOL) surveys include generic measures that enable comparisons across conditions and measures that focus more specifically on one disease or condition. We evaluated the psychometric properties of German- and English-language versions of survey scales representing both types of measures in samples of current and former smokers. METHODS: TQOLITv1 integrates new measures of smoking-specific symptoms and QOL impact attributed to smoking with generic SF-36 Health Survey measures. For purposes of evaluation, cross-sectional data were analyzed for two independent samples. Disease-free (otherwise healthy) adults ages 23-55 used a tablet to complete surveys in a clinical trial in Germany (125 current and 54 former smokers). Online general population surveys were completed in the US by otherwise healthy current and former smokers (N = 149 and 110, respectively). Evaluations included psychometric tests of assumptions underlying scale construction and scoring, score distributions, and reliability. Tests of validity included cross-sectional correlations and analyses of variance based on a conceptual framework and hypotheses for groups differing in self-reported smoking behavior (current versus former smoker, cigarettes per day (CPD)) and severity of smoking symptoms in both samples and, in the German trial only, clinical parameters of biomarkers of exposure. RESULTS: Tests of scaling assumptions and internal consistency reliability (alpha = 0.71-0.79) of the smoking-specific measures were satisfactory, although ceiling effects attenuated correlations for former smokers in both samples. Correlational evidence supporting validity of smoking-specific symptom and impact measures included their substantial inter-correlation and higher correlations (than generic measures) with smoking behavior (favoring former over current groups) and CPD in both samples. In the German trial, both smoking-specific measures correlated significantly (p \u3c 0.05) with all four biomarkers. QOL impact attributed to smoking correlated with the SF-36 mental but not physical summary measures in both samples. CONCLUSIONS: German- and English-language TQOLITv1 surveys have comparable and satisfactory psychometric properties. Cross-sectional tests, including correlations with four biomarkers, support the validity of the new smoking-specific measures for use in studies of otherwise healthy smokers. Smoking-specific measures consistently performed better than generic QOL measures in all tests of validity

Using the bootstrap to establish statistical significance for relative validity comparisons among patient-reported outcome measures

BACKGROUND: Relative validity (RV), a ratio of ANOVA F-statistics, is often used to compare the validity of patient-reported outcome (PRO) measures. We used the bootstrap to establish the statistical significance of the RV and to identify key factors affecting its significance. METHODS: Based on responses from 453 chronic kidney disease (CKD) patients to 16 CKD-specific and generic PRO measures, RVs were computed to determine how well each measure discriminated across clinically-defined groups of patients compared to the most discriminating (reference) measure. Statistical significance of RV was quantified by the 95% bootstrap confidence interval. Simulations examined the effects of sample size, denominator F-statistic, correlation between comparator and reference measures, and number of bootstrap replicates. RESULTS: The statistical significance of the RV increased as the magnitude of denominator F-statistic increased or as the correlation between comparator and reference measures increased. A denominator F-statistic of 57 conveyed sufficient power (80%) to detect an RV of 0.6 for two measures correlated at r = 0.7. Larger denominator F-statistics or higher correlations provided greater power. Larger sample size with a fixed denominator F-statistic or more bootstrap replicates (beyond 500) had minimal impact. CONCLUSIONS: The bootstrap is valuable for establishing the statistical significance of RV estimates. A reasonably large denominator F-statistic (F \u3e 57) is required for adequate power when using the RV to compare the validity of measures with small or moderate correlations (r \u3c 0.7). Substantially greater power can be achieved when comparing measures of a very high correlation (r \u3e 0.9)